The present invention relates to a system and method for pixel compression to reduce data rate and memory storage requirements in real-time processors. More particularly, it relates to non-integral pixel compression for second generation FLIR sensors.
Pixel compression is employed in present FLIR systems to reduce data rate and memory storage requirements of real-time processors used to analyze imagery. These systems typically consist of a vertical, staggered line array of 120 or 180 detector elements as illustrated in FIG. 2. The target, or image to be detected, is scanned across the detector perpendicular to the line array in the horizontal direction. Each detector element is sampled periodically as the scan proceeds, creating an array of pixels. The sampling frequency is such that each detector is sampled at least two times for each horizontal detector width. Higher sampling rates which yield two, four or even 10 samples per detector are not uncommon.
In most line array sensors, the situation is further complicated by the fact that the effective detector aspect is not square. The vertical dimension is typically 1.75 to 2 times greater than the horizontal dimension. Thus the raw pixels represent rectangular solid angle samples of the image. The rectangular nature of the sample is undesirable from the view point of machine processing of the image.
The operation of Pattern Recognizers and video trackers must be invariant to the aspect or viewing angle of the scene in order to achieve consistent performance. Thus if rectangular pixels are sampled, aspect angle changes would induce distortion in the scene. Pixel compressing is employed in the horizontal direction to produce square pixels for the machine processing functions. This is particularly true in multiple use sensors where the sampling rate is set by other requirements, most notably the one to produce an image pleasing to a human operator.
The usage of pixel compression in line array sensors is best understood by way of example. Consider a staggered line array detector consisting of two columns of 90 elements each. Each column is separated by an integer multiple of horizontal detector widths. The staggered nature is actually irrelevant to the sampling. The staggering can be removed electronically; but by separating the columns, a detector array, without gaps in the vertical coverage, can be manufactured. Effectively, 180 contiguous vertical samples are taken at each horizontal location.
Now in this example, the vertical dimension will be 1.75 times the horizontal dimension. Each element will be sampled 3 times as the scene is scanned across one horizontal detector width. FIG. 2 illustrates the coverage of a single detector element for several samples. The consecutive samples have been offset slightly in the vertical direction for clarity of presentation. If the scanning continues for a width equal to the total vertical dimension of the line array, then an array of 180.times.945=170,100 pixels results. If this is scanned in 12.5 msec., then the data rate is 13.6 Mhz. Note the comparative horizontal coverage of one, three and four pixels, as shown in FIG. 2.
Horizontal pixel compression can be applied to each channel individually. If three horizontal samples are combined (by averaging) into a single sample, then the effective size of a pixel is 1.75.times.1.667 angular units, yielding a 0.95 aspect ratio. The resulting array consists of 180.times.315=56,700 pixels and is generated with an equivalent data rate of 4.536 Mhz. If four samples are compressed into one, then the size is 1.75.times.2.0, yielding a 1.143 aspect ratio. The resulting array of pixels is 180.times.236=42480 and is generated at an effective data rate of 3.402 Mhz. The data rates and pixel totals resulting from pixel compression are more consistent with the capabilities of existing real-time programmable signal processors. Note that although the aspects are not square and thus do not yield an aspect ratio of 1.0, the aspect distortions are within acceptable limits. Thus pixel compression can be used to compensate for detector aspect distortion as well as reducing data rates and storage requirements. It is true that resolution will be lost in the horizontal direction, but to utilize the full resolution of a 180 by 945 array, processors beyond those which currently exist would have to be created.
A new generation of IR detector arrays comprise a detector configured as in FIG. 3. This is a 480.times.4 focal plane array With TDI (time delay integration). The latter is a feature which synchronizes the horizontal sampling of the four elements in the horizontal direction such that, as the scene is scanned, the signals of the 4 elements are added as each reaches the same angular position in the scene. When implemented properly, the 4 TDI elements are not germane to this discussion and the array can be considered an effective 480.times.1 array, with twice the signal to noise ratio afforded by an array of a single horizontal element. Other properties, however, are also very important. Unlike first generation devices, the aspects of detector elements are square. Also there is vertical interlace since the second column is offset by one-half an element in the vertical direction, from the first. Furthermore, there is comparatively (as compared to first generation devices) little dead space between detector elements. Each column, to a first approximation, can be thought of as providing contiguous coverage in the vertical direction.
The array elements are physically smaller than those typically encountered in first generation systems. They subtend a smaller solid angle, typically half that of conventional line arrays. This is important when considered in terms of total system resolution. The minimum resolvable angle of any FLIR system is determined primarily by the diameter of the objective lens. This establishes the blur circle of the sensor, which represents the angular spread of a point object at infinity induced by the sensor optics.
FIG. 4 illustrates the blur circle of a typical optical system as compared to the horizontal width of a first and second generation detector. The width of the blur circle is such that about 85% of the point energy is contained within the angle subtended by the first generation detector. On the other hand, only about 30 to 40% of the blur circle energy falls on each second generation detector. Thus the newer arrays are oversampled from a spatial point of view.
One possible way to fix this is to increase the aperture diameter. This decreases the blur circle width and by making the aperture large enough, the same relative percent of energy falling on the detector could be maintained. This is impractical, however, since the size of the objective lens is limited by system size, weight and cost constraints.
The oversampling could also be dealt with by simply increasing the number of samples. This too, however, is not desirable. It is useful to compare the sampling raster for a second generation array with the example of the first generation detector's sampling raster discussed earlier. For example, a 480.times.4 second generation array would generate a typical sampling raster of 480.times.960 elements. In the typical application, each dimension of the detector element of the second generation array is 0.8 of the horizontal angular subtense of the first generation array. This permits reduction of the electronic, horizontal sampling to two samples per horizontal detector width. Given the reduction in detector size, the 480.times.960 raster would cover 2/3 of the angular dimension, in each direction, of the first generation example. To cover the same angular area, (which would require more detector elements) the raster would have to be 720.times.1440 pixels. In either case, the number of pixels 460,800 (1,036,800) and the corresponding data rates 36,864 Mhz (82.944 Mhz.) are too high to be accommodated by existing array processors or those that will be available in the near future.
Despite the higher data and sampling rates required, the second generation detectors have advantages which can improve system performance. These include inherent vertical interlace and square pixels, as well as lower inherent noise. These are properties which are very desirable when the system is used to form an image for a human operator. Thus it would be desirable to find a way of reducing the number of samples used in a machine processor without affecting the image seen by an operator.